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ABSTBftCT 



Ihe objective of this study is to develop and test two 
new approximate solution techniques for a moving target 
problem in discrete time and space where both the searcher 
and the target have constraints on their paths. The first 
technique is an application of the Local Search Method and 
the second technique is an application of the Franx-Wolfe 
Method. The motivation for looking at approximate methods 
is that the problem is MP-complete and optimal solution 
techniques become impractical for large size problems. 
Experiments showed that the Local Search Method approach is 
an efficient technique for obtaining approximate solutions. 
However, the Frank-Wolfe Method approach does not perform 
well for the problem. 
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I. I HTBODO CTIOH 



A. I BE PROBLEH 

Ihe problem considered here is the search for a moving 
target in discrete time and space, where there are 
constraints on the searcher's movement. The target is 
assumed to move among a finite set of cells 
according to a specified , Markov transition matrix 

Z={z(i, j)}- The target's motion is independent of the 
searcher's actions. The transition matrix is known to the 
searcher as well as the initial distribution of the target 
over the search ceils. In each time period one cell is 
searched. The searcher is able to move from his current 
cell, say i, to a cell which must be selected from a given 
subset C (i) of the set C. Thus, the search cell in a given 
time period must be within some specified neighborhood of 
the search cell in the previous time period. If the 
searcher and the target are in the same cell i, the target 
will he detected with probability g (i) . If the target is 
not in the cell which is searched, it cannot be detected 
during the time period. The objective of the search is to 
find a T-time period search path which minimizes the prob- 
ability of non-detection (PND) . 

B. BACKGROUND 

The problem of searching for a moving target in discrete 
time and space has received considerable attention. The 
problem is difficult because of its complexity. Truimel and 
Weisinger [Ref. 1 ] proved that optimal constrained path 
search problem for a stationary target in discrete time and 
space where the time horizon is finite and the measure of 
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effectiveness is PND, is NP-Complete. Since the stationary 
target problem is NP-Complete, the moving target problem is 
also NP-Complete. For more information on NP-complete prob- 
lems, see (Ref. 2]. The optimal solution techniques which 
have been used to solve the constrained path moving target 
problem are: 

1. lotal enumeration 

2. Ihe dynamic programming technique of Eagle [Ref. 3]. 

Ihe total enumeration method is complete enumeration of 
all possible search paths. The disadvantage of this method 
is the exponential increase in computer time as the problem 
size increases. However, it has the advantage of requiring 
very little computer storage. Eagle formulates the problem 
as a Partially Observable Markov Decision Process (POMDP> , 
and the method requires extensive computer storage, but 
finds solutions more quickly than the total enumeration 
method. 

Since the problem is NP-Complete, it is difficult to 
solve large problems. For large problems, heuristic or 
approximate solution techniques have been proposed. Such 
methods were developed by Stewart [Ref. 4] and Eagle 
[Ref- 5]- 

Stewart developed a branch-and-bound algorithm for the 
solution of the problem. However, optimality cannot be 
guaranteed because exact bounds cannot be obtaxned. 
Nevertnless, Stewarr used one-dimensional a random walk as a 
test problem and demonstrated that the algorithm performs 
efficiently. 

Eagle's approximate procedure is a simpler version of 
his optimal dynamic programming method. Eagle used moving 
horizon policies and also looked for a lower bound for the 
minimal PND. Eagle's computational experience with 
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two-dimensional search problems showed the m-time period 
moving horizon policies (m-TPMH) do not necessarily perform 
better as m increases. Nonetheless, the method does show 
some promise. The m-TPMK policy selects as the next search 
that cell which would be optimal if m-time periods remained 
in the problem. 

Eecently Eagle and Yee developed a new approximate solu- 
tion method by using the convex simplex method [fief. 6]. 
Two-dimensional computational experiments showed that the 
method performs well. 

In this thesis two new approximate solution techni'^ues 
will te studied and tested. The first method is a heuristic 
approach which is an application of the Local ( Neighborhood) 
Search. The second technique is an application of the 



Frank-Wclfe method 



II. F OBaOL AT ION ^ THE PROBLEM 



Id this chapter we will formulate the problem as a 
Donlirear program. 



a. DEFINITION OF SYMBOLS 



T ; Number of time periods. 

N ; -Number of cells. 

q (i) : Prob {detecting the target i target and searcher 

are in cell i} . 

Z = ( 2 (i,j)} is an NxN Markov transition matrix where the 
(i,1)th element is the probability that the target in cell 
i at time period t will move to cell j at time period t+1. 

x(t) = {X { 1,t) X (N ,t) } represents the searcher 
probability distribution at time t. That is, x{i,t) 
denotes the probability of the searcher being in cell i at 
time period t. At the start of the search, the searcher 
cell IS assumed to be known with certainity, i.e., x{1) 

consists of 1 one and all others zero. 

Pt'bi = {P(1#t)... »P(N,t)} represents the target prob- 

ability districution at time t. That is. p(i,t) is the 
probability target is in cell i at the begining of time 
period t without being detected by the previous t-1 
searches. The initial target distribution P{1) is assumed 
to be known by the searcher. 

R(i.j,t) is the probability that the target is in ceil i 
without being detected by the previous t-1 searches and 
the searcher is in cell j. 

S <t) = (S(i,j,t)) is an NxN Markov transition matrix for 

the searcher. The (i,j)th element is zero if subset C(i) 
does not include j (patn constraints) . 

S = {S ( 1) . . . . . ,S {I- 1) } is called a search plan which 
includes all possible searcher motions for T-time period, 
given that the searcher starting cell is known. 



B. NIP FORMULATION OF THE PROBLEM 

New we need to compute x(t)*s and R(i,j,t)'s. Since 
x(1) is known, we can determine x(t) by the Markovian motion 
as shewn below; 



x{t) = x(t-l) S(t-l), t=2,...,T. 

1 1 



The E(irj,t) consists of two Markov processes. Note that 
is assumed to be known. So, the fi(i,j,t) can be 
computed as follows: 



R(i,j,t) = {1-g (i) d (i> j)} B(k>l,t-1)z(k^i) S(l> j^t-1) 

k,L 






, N and t=2,3,.... ,T 



where d(i,j) is one if i equals j, zero otherwise. 
Therefore, given P(1), q(i)# S, and Z, the T-time prob- 

ability of nondetection is 

PND =^^B(i,j*T), i# 3~ ^ # 2 , . . . , N. 

He want to fina the search plan S which minimizes the 
T-txme period PND, subject to the path constraints. The 

final NLP form of the problem is given in Table 1. The 

following properties of the NLP were proven by Eagle and Yee 
[Ref- 6]. 

Proposition 1: The minimum of the NLP is achieved by a 

deterministic search plan.. A deter ministic search plan 
consists of ones and zeros. 

Proposition 2: S is a deterministic search plan if and 

only if S is an extreme point of the linear constraints. 

Proposition 3: The objective function is linear in S 

wnen constrained to any edge of the simplex. 
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TABLE 1 

HLP FOBHOLATIOH OF THE PfiOBLEfl 

miB PBD 
suhject to: 

S(t) 1=1 , t=1,...,T-1 

S(t) > 0 , t=1, ,T-1 

S(i»jft) = 0 f t— f m m .^T~1 

i=1, N 

j ^ C(i) 
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III. TJJ LOCAL SZAB^ ALG OB JTHM 



The optimal solution technigues for hard combinatorial 
optimization problems require more computer time when the 
size of the problem increases. Therefore, they become 
difficult to implement for large size problems. One of the 
most succesful methods of attacking them is the Local Search 
or Neighborhood Search. However unsatisfying mathemati- 
cally, this approach is certainly valid in practical sxtu- 
ations. The main shortcoming of the method is that there is 
no criterion to tell whether the obtained solution is a 
global optimum. The local search method is most applicable 
for optimization in two main types of problems: 

1. When the analytic relationship of the independent 
variables and the objective function is not known, 
but the value of the objective function can be evalu- 
ated at individual points by experiments. 

2. When the analytic form of the objective function is 
known but there is no finite algorithm for obtaining 
the extreme value (s) in a closed form. 

The efficiency of the method, relative to another, is 
defined in terms of a suitable cost function such as number 
of points or experiments required for localizing the optimum 
within a specified range. One of the best known illustra- 
tions of this technique is the traveling salesman problem 
<TS?) where the objective is to find the shortest path 
passing through each of a number of points exactly once and 
then returning to the starting point. For other examples 
refer to [ Ref. 7 ]. 

Since the problem presented in the previous chapter is 
NP-complete and the current optimal solution techniques are 
impractical for large size problems, we can try to apply the 



local search method as a heuristic approach for good 
approximate solutions- 

A. IBE ALGOeilHH 

First, we will define some concepts in order to intro- 
duce general local search algorithms and tnen apply this 
method to the constrained path moving target search problem. 

Definition 1: An instance of an optimization problem is a 

pair (F,c) where F is any set, the domain of feasible 
points, and c is the objective function, a mapping 

c; F — > 

Definition 2: Given an optimization problem with instances 
(F,c) , a neighborhood is a mapping 

N2 F — > 

defined for each instance. 

The general local search algorithm is described as 
follows: Given an instance (F,c) of an optimization problem 

(minioizing) and an initial feasible point s, we search the 
neighborhood N (s) for an n£N(s) with c (m) < c{s)- So, we 
start with an initial feasible point s and attempt to 
improve the objective function by searching the neighborhood 
of s. If an improvement is found, another search is made in 
the neighborhood of the improved point. This continues 
until no improvement can be made. 

Since we know that the solution of the NLP is an deter- 
ministic search plan (Proposition 1) and it is assumed that 
the searcher starting cell is known, we can reduce our 
feasible region to search paths s= (s ( 1 ),..., s (T) } , where 
s(t) is the searcher ceil at time period t. Therefore, we 
can start with an initial feasible search path s, an objec- 
tive function value END (s) , and attempt to find a search 
path m in a neighborhood of s such that PND(m) < PND (s) . *e 
define the k-change neighborhood of search path s as all 
feasible search paths m which are identical to s except in k 
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consecutive tine periods. So, the algorithm for the problem 
becomes : 



Step 1. Start with an initial feasible search path s. 

Step 2. Find the all possible k-change neighborhood paths 
by removing k consecutive cells from s (keeping the others 
unchanged) then replacing them with k new cells satisfying 
the path constraints. 

Step 3. Find the k-change neighborhood path m with the 
minimum PND (m) . 

Step 4. If PND(m) < PND(s), set s=m and go to step 2. 
Otherwise, STOP; current s is k-optimal. 



The most difficult task in any application is to decide 
of the proper value of k for the k-change policy. Selecting 
the rignt k leads to very effective heuristics for the T3P 
fHef. 7]. We will test the present algorithm for k=1,2 and 

3. 



B. INITIAL FEASIBLE SEABCH PATH 

The algorithm reguires two stages. First, determine the 
initial feasible starting search path and then apply the 
procedure to it. The initial feasible point may be chosen 
randomly or it may be carefully constructed. In practice we 
need a guick and a good approximate solution. We do not 
want to be far away from the optimal when we start. In this 
study we will use the modified myopic policy (HMP) to deter- 
mine an initial feasible starting path. The myopic policy, 
which gives optimal solution for a stationary target prob- 
lems, always searches in the next time period t, the cell i 
which has the largest probability of detecting the target, 
(I (i) F (i#t) } . The MHP is defined as follows. 

For the next time period t, searcher moves from his 
current cell i to that accessible cell j which has tne 
largest g(j)p{j,t) value. If all accessible cells j have 
identical values of q(j)p(j,t), then the searcher will move 
to the accessible cell which has the minimum Euclidian 
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distance to the cell k having the largest g(k)p{k,t) value. 
If the Euclidian distances are equal, these ties are broken 
randomly. There is probably a smarter technique for 
breaking ties. 

The MMP is a heuristic construction meant to produce a 
good starting path, because the neighborhood k-change proce- 
dure may not be very powerful. We will also test the 
(k- 1 ) -optimal solution as a starting path for the k-change 
policy. 
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IV. COMPOTATIONAL EXPEBIEHCE 

Our computational experience is based on two-dimensional 
search problems. All experiments presented in this chapter 
were pertormed on an IBM 3033 mainframe computer using 
Fortran 77. The Local Search Algorxthm was used to solve 
three 10-time period search problems, where N=9 , 25 and 49. 

For each problem, values of k=1, 2 and 3 were used. The 

constraints on the searcher and the tar^^et paths are defined 
as; the searcher (target) is able to move to the cell previ- 
ously searched (occupied) plus all adjacent cells. Cells 
are adjacent if they share a common side. For example, in 
Figure 4.1 the adjacent cells of cell 1, 4 and 5 are: 

C(1)={1,2,4) , C(4)=(1,4,5,7) , C (5) = {2, 4 , 5, 6 , 8) 



The target transition matrix is; the target remains in the 
previously occupied cell with probability 0-4, and moves to 
the adjacent cells with probability 0.6/m where m is the 
number of adjacent cells- The probability of detection is 
1.0 for each cell, i.e., g(i) = 1.0 ,i=1,...,N. 



A, 3X3 PROBLEM 
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Figure 4.1 



9 Cell Search Grid, 



Id this problem the target moves among the 9 cells shown 
in Figure 4.1. The searcher starts in cell 1 and the target 
in ceil 9, i.e. , initial target distribution is that the 
target is in cell 9 with probability 1.0 and in the other 
cells with probabiliy 0- The 10-tiae period search results 
are shown in Table 2. An optimal solution was obtained for 
all k=l, 2, 3. However CPO time increases exponentially as 
k increases. As can be seen, the MM? solution is very close 
to the optimal. The Fortran code for this problem is given 
in Appendix A. 



1 














1 








TABLE 


2 












3x3 PROBLEM 


RESULTS 












Search Path 




PND 


CPU 

T ime (sec) 




Myopic 


4 


5 


8 5 6 5 8 5 


6 


5 0.2328 






k = 1 


4 


5 


8 9 6 5 8 5 


6 


5 0.2214 


0.5 1 




CM 

II 


2 


5 


8 9 6 5 8 5 


6 


5 0. 2214 


3-10 




k = 3 


2 


5 


8 9 6 5 8 5 


6 


5 0.2214 


26, 1 8 










OPTIMAL PND 


— 


0. 2214 







B. 5X5 PHOBIEM 

In this example the target and the searcher moves among 
25 cells (Figure 4.2). The searcher starts in cell 1 and 
the target starts in cell 13 with certainty. The 10-time 
period search results are shown in Table 3. 

We found an optimal solution only for k=3, but solutions 
for k = 1, 2 are close to the optimal. The k=2 solution 
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Figure 4.2 25 Cell Search Grid. 
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differs from tne optimal solution only in the fourth digit. 
Similar to the 3x3 prcbleu the MMP solution is close to the 
optimal and CPU time increases exponentially. 

C. 7X7 PROBLEM 

In this proniem the target and the searcher moves among 
49 search ceils shown in Figure 4.3. The searcher starts in 
cell 1 and the target starts in ceil 17. The 10-time period 
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search results are shown in Table 4. Me obtained the 
optimal solution for only k=3. The percentage difference 
between the MMP solution and the optimal solution is only 
1.2 %. For k=1 and 2 same result were obtained. 

Exponential increase in CPU time is more steep than the 
previous problems. 

In addition, the total enumeration method was used to 
find ail optimal paths for each problem. The results are 
shown in Table 4. 
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Figure 4.3 49 Cell Search Grid. 



In tact, for the 5x5 and 7x7 problems we increaseo the 
search area of the 3x3 problem with toe same targc.t and 
searcher starting ceils and the target transation matrix. 
For each proniem a similar pattern was observed for optimal 
paths. Specifically, the searcher goes to the target's 
starting cell as soon as possible, tnen starts to search 
outside of this cell similar to the expanding square search. 
Graphical representation of exponential increase in CPU time 
as k increases, is shewn in Figure 5. 1- 
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Figure 4.4 CPU Time vs k-change Policy. 



D. HCDIFIED MYOPIC SOIOTION HESOLTS 

The reason for using the MMP was to find a near optimal 
starting solution. Ihis objective was met in the problems 
shown so far. In these problems, the mean position of the 
target does not move, given no search. In order to examine 
the possible effects of a moving mean target position, two 
5x5 search problems were solved with different transition 
matrices, initial target distributions, and detection 
probabilities. 

1 • lASii Target Pr oblem 

In this problem the target starts in cell 13 and the 
searcher starts in cell 1. The target transition is sach 
that the target stays in the previously occupied cell with 
probability 0 and moves to the adjacent cells with prob- 
ability 1/m where m is the number of adjacent cells. The 
detection probability is 1.0 for each cell- The 10-time 
period search results are shown in Table 5. 



i 1 

TABLE 5 

5x5 FAST TARGET PROBLEM RESOLTS 

CPU 

Search Path PUD Time (sec) 



Myopic 


2 


7 


8 


8 


13 


18 


13 


18 


13 


8 


0.4770 




k = 1 


2 


o 


7 


12 


17 


18 


19 


14 


9 


8 


0.374 2 


1.8 


k = 2 


2 


7 


7 


12 


17 


18 


19 


14 


9 


9 


0.3742 


10. 1 9 


k = 3 


2 


7 


7 


8 


13 


12 


17 


18 


19 


14 


0.3779 


67.93 



OPTIMAL PND = 0.3742 
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In tills case the MMP did not perform as well as it 
did in the previous 5x5 problems. An optimal solution was 
obtained for k=1 , 2. Additionally, k=3 policy did not reach 
the optimal solution while requiring more CPU time. 

2. Escaping Tar get Problem 

For this problem, the target starts in cell 21. It 
then stays in the current cell with probability 0.1 and 
moves one cell up or to the right with probability 0.45. If 
it reaches one of the top boundary cells ( 1, 2, 3, 4 ) or 
one of the right boundary cells ( 25, 20, 15, 10 ), then it 
will move one cell to the right or one cell up with prob- 
ability 0.9 and it remains in the current cell with prob- 
ability 0.1. When it reaches cell 5 it will stay there 
forever and is assumed to have escaped. That is, the prop- 
ability of detection in this cell is 0, while for the other 
cells the probability of detection is 1.0. The 10-time 
period search results are shown in Table 6. The bMP path 
has significantly greater PND than the optimal path even 
though it differs in only one time period from an optimal 
path. Optimal solutions were obtained for k=l, 2, 3- 

For the last two problems all optimal paths were 
obtained by total enumeration and are shown in Table 6. In 
the fast target problem, the optimal search path placed 
search effort around the targets starting cell. If target 
moves in a specific direction (escaping target problem) this 
reduces the number of possible optimal paths. In this case, 
optimal paths put a barrier in front of the target. 

E. INCBEASING THE HOHBEfi OF TIME PERIODS 

To examine the effect of increasing the number of time 
periods on CPU time, 3x3 problem was solved for 15, 20, 25 
and 3C time periods. The graphical results are shown in 
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TABLE 6 

ESCAFIBG TARGET FfiOBLEM RESOLTS 



Myopic 


6 


1 1 


Search : 
12 12 13 


Path 

14 


15 


15 


10 


10 


PND 

0.2588 


CPU 

Time (sec) 


k = 1 


6 


1 1 


11 


12 


13 


14 


15 


15 


10 


10 


0.1530 


0-68 


k = 2 


6 


11 


11 


12 


13 


14 


15 


15 


10 


10 


0. 1530 


3.03 


k = 3 


1 


6 


11 


12 


13 


14 


15 


15 


10 


10 


0. 1530 


1 8,4 6 



OPTIMAL PND = 0.1530 



TOTAL ENUMERATION RESULTS 2- 
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17 


18 


19 


14 
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Figure 4.5 for Jc=1, 2, and 3. For T=30 and k = 3 prograa was 
stopped at 3600 seconds CPO time. The increse in CPU time 
for k=l was considerably less than for k=2, 3. We can say 
that k=1 policy was the most CPU TIME / (1-PND) effective. 

Fcr the presented problems we also tested the 
(k-1) -change optimal solution as a starting path for the 
k-change policy where k=2. But starting with this alterna- 
tive path did not improve the solution obtained by using a 
MMP starting path. 
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Figure ^*.5 Time Periods vs CPU Time 



V- FBAMK-BOLfE ALGORITHM 



Another method applicable to the problem is the 
Frank-Wolfe (F-H) algorithm. The main motivation of 
selecting this algorithm is that this method implicitly 
examines all extreme points of the convex polyhedron, rather 
than just the k-change ones. In 1956 Marguerite Frank and 
Philip Wolfe proposed this method for solving non-linear 
programs having a convex differentiable objective function 
and linear constraints. Since our objective function is not 
convex this method will determine an approximate solution to 
the problem. 

The F-W algorithm is a feasible direction method and may 
simply stated as follows; Given a feasible point at any 
iteration, an improving feasible direction is determined by 
linear program (LP) . The objective function of the LP is 
formed by using a linear approximation (first order Taylor's 
expansion) of the objective function of the NLP and the 
constraints are the original linear constraints of the NLP. 
Then the step size is determined by a line search. ihe 
algorithm continues until no direction is found to improve 
the objective function. For further information refer to 
[Ref. 8]. 

A. AfPLICATIOM OF F-H METHOD 

Given S, X (t) , P (t) , g (i) , and Z, the objective function 
can be written for each time period t as follows: 



PND 



(S) p (i,t) {1-g (i)x (i, t) } z (i, j) V (j ,t+1) 



i, j=1, ....... N and t=1,2. 
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where is the probability that the target is in cell i 
at the beginning of time period t and will not be detected 
by the remaining searches. Be can compute v(i,t)*s in a 
nacJcward recursive manner as fellows: 



V (i,T+1) =1- 0 , i=1,. .. ,N 



V <i,t)= ^ {1-g(i) x(i,t)}z(i,j) v(j,t+1) 

iecu) 

and .« .. 



Since P(1) is known, the p(i,t) *s can be computed by the 
forward recursion 

P(3»t+1) =^p(i,t) {1-q(i)x(i,t)}z(i,j) 

L 

i»j=1»2, -.o--,N and t=l,2,-....,T-1 

For a given search plan S the p(i,t) and v(i,t) will be the 
same for each time period. So, we need to compute them once 
for each search plan S. 

Since x(j,t)= x (i, t- 1 ) S (i, j, t- 1) , then the gradient 
vector for time period t can be computed from the following: 

^ PNO(s) ■ 

I (i.jrt) =-x(i,t) p( j,t+1) 2 g ( 3 ) z( j,k) v(k,t + 2) - 

W£C(j) 

i» 3 = 1 ....... N and t= 1, 2,.. , I 
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Then th€ feasible direction of descent is found by solving 
the Lf subproblein shown in Table 7. As we explained before 
the constraints of the LP must be the same as the NLP's 
constraints. Note that the IP's constraints in Table 7 is 
another way of expressing the constraints in Table 2. The 
solution of this LP subproblems is trivially found as 
follows; For each i and t, choose j C (i) witn minimum 
Y(i,j,t), call it j then set M(i, j ,t) = 1 and M(i,j,t)=0 for 
i = D • 



TABLE 7 
LP SOBPEOBLEM 



min 
subject to; 



a T*^3f(i,j,t) M(i,j 

ti T 



, t) 



rt(i,j,t) = l , 
3T£C'«) 

M(i, j,t)^0 , 



i=1 , . . , , N 

t=U.. .,T-1 



i=l , , - . , N 
t=1,.. T-1 



This solution is an extreme point of the linear 
constraints. Therefore, solution of the LP is deterministic 
search plan which defines a uaigue search path m. Since 
we are only interested in deterministic search paths, we 
choose the step size as 1 if PND(m) < PND(s) and zero other- 
wise, ie., tne process is terminated when PND(m) > PND(s) . 
So, the second algorithm becomes: 

Step 1. Select an arbitrary feasible search path 
s={s(1),..,s{T-1}. 



step 2. Determine a new search path m by solving the LP 
sat problem. 

Step 3. If PND(m) < PND(s) set s=m, go to step 1. 
Otherwise, STOP, 



B. CCMPUTATIOHAL EXPEBIEHCE FOR F-H METHOD 

We applied the algorithm to the 3x3 problem (defined in 
Chapter 4) with 30 randomly selected starting search paths 
(myopic search path included). The algorithm did not reach 
the optimal solution. For 7 of the 30 paths (myopic path 
was one of them) the algorithm did not improve the starting 
solution. The closest solution to optimal (0.2214) was 
0.2328 which is the myopic path result. The maximum and 
minimum CPU times were 0.04 and 0.11 seconds, respectively. 
The results are shown graphically in Figure 4.4. The 
Fortran code of the algorithm for the 3x3 problem is given 
in Appendix B. 
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RANDOM STARTING SEARCH PATHS 



VI. COHClOSIOHS 



H€ have investigated the effectiveness and computational 
efficiency of the Local Search Method and the FranJc-Wolfe 
Method for several two-dimensional search problems. 

The heuristic Local Search Method appeared clearly 
satisfactory for the examined search problems. However# the 
exponential increase in CPU time will limit the use of the 
method for higher k values. Also increasing k does not 
necessarily improve the solution. This procedure appears to 
be useful because it allows a trade off between solution 
accuracy# i.e.# closeness to the optimal search path and CPU 
time. for further analysis this method can be tested for 
different problems# k-change policies and number of txme 
periods. 

The Frank-Wolfe method approach did not perforr well for 
the problems examined. An alternative way of improving 
efficiency of this algorithm may be to perform a line search 
after determining the feasible direction of descent. Bat 
this would increase the CPU time. Both methods are also 
applicable to Brown’s original problem £ Bef . 9] and they may 
be efficient for this problem because of its convex objec- 
tive function. 
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APPENDIX A 

FORTRAN CODE OF LOCAL SEARCH ALGORITHM 



C 

C 

C 

C 

C 

C 



C 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

100 

103 

101 

102 

10 ^ 

c 

c 

c 

15 



25 

c 

c 

c 

35 



C 

c 

c 

999 

353 



C 

C 

C 

C 

C 

C 



^ METIN 5AGAL ^ 

X THIS PROGRAM APPLIES LOCAL SEARCH 
X ALGORITHM TO 3X3 PROBLEM FOR K=l. 



INTEGER T.NN,P(900,30) , TT ( 50 ) , S ( 50 ) , C( 25 , 25 ) 
INTEGER AA(9,5),IN(9) 

REAL PP(20) ,PB(50) ,BB(50) ,XMIN,PND 






^ INPUT 

T=TIME PERIODS 
NN=«OF SEARCH CELLS 
S^STARTING SEARCH PATH 
ir^ = fiOF ADJACENT CELLS 
A^ADJACFNT CEl LS 
C^CONSTRAINT MATRIX 



-k^k Ik ii It * k i 



kkkkk 



READ(5,100) T,NN 
F0RMATC3I2) 

READ(5,103) (SCI), 1=1, T) 

FORMATC 1 111 ) 

READ(5,101) ( INC I) , 1 = 1 , NN) 

F0RMAT(9I1 ) 

READ(5,102) AA(I,J),J=1,5),I=1, NN) 

FORMATC 511 ) 

READC5,109) CCCCI,J),J=1, UU) , 1=1 , NN) 

FORMATC 912) 

TARGET INITIAL PROB. DISTRIBUTION 

DO 15 K=1 ,NN 
PBCK)=0 
PBC 9 ) = 1 
DO 25 K=1 , NN 
BBCK)=PBCK) 

kkkkkkkkk FltJD PND FOR STARTIfJG PATH S kkkkkkk 

DO 35 1^1,10 
ZC I )=SC I + l ) 

CALL PROBCZ,PB,T,NN.BB,PND) 

XMlN = Pr^D 

kkkkkkkitkk FIND NEIGHBORHOOD PATHS it it # Jt it j( 

MM^M=0 
TT = 2 

K ^ I fU S C T T - 1 ) ) 

DO 10 IM ,K 

ZC TT-1 )=AAC SC TT-1 ) , I) 

kkk kkkkkkk CHECK PATH COf^STRAINTS k k kkk k k k k k k k 
IFCCCZCTT-1 ),SCTT+1 ) ) EQ.O) GO TO 10 

FIND PND FOR NEN PATH k k k k k k k k k k k k k 
CALL PROBCZ, PB, T , U\\ , DB , PND) 
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IFCPtJD. LT . XMIN) IHEU DO 
M = M+ 1 

C 

C KEEF’ the best path ArJD PND 

C 

XM1N=PND 
DO 30 IB=1,10 

30 PCM, IB)=Z( IB) 

N = 0 

END IF 

DO ^0 IK=1 ,10 
^0 Z(IK)=S(IK+1) 

10 CONTINUE 
C 

C CHANGE THE NEXT CELL 

C 

TT=TT+1 

IFCTT.LE.il) GO TO 333 
C 

C IF THERE IS AN IMPROVEMEtJT , THEN SET S 

C ^TO THE BEST NEIGHBORHOOD PATH AND START AGAIN 
C 

IFCMM.GT.O) THEN DO 
SC n = i 

DO 50 I=Z. 1 1 

SC I)=PC 1 . I-l ) 

50 Z( I-l )=PC 1 , I-l ) 

GO TO 999 

END IF 
C 

C IF THERE IS NO IMPROVEMENT, STOP ^^^***^^ 

C 

PRINT ,♦ OPTIMAL PATH IS 

HRlTECb , 200 ) C SC I) , 1^2, 1 1) 

200 FORMATC10I2) 

PRINT, 'OPTIMAL PND IS*,XMIN 
STOP 
END 
C 

C ^ CALCULATES PND * ** * ^ * * * * it 

C 

SUBROUTINE PROBCZ, PB , T . NN, BB , PND) 

REAL AC50),PBC50),PND,BBC50) 

INTEGER NN,T,ZC50) 

KP=T-1 

DO 71 K=1 ,KP 
PBCSCK) ) = 0 

ACl) = .Ai^PBCl)+.2itPBC2)'^.2*PBC^) 
AC2)=.3»PBCl)+.9itPBC2)+.5^PBC3) + .15*PBC5) 

AC3) = .?^PBC2)+.9i(PBC3)4.2i^PBCb) 
AC9)=.3>PBCl)+.AitpB(9)+.15^PBC5) + .3^PBC7) 

AC5) = .?i^PBC2) + .2itpBC9)4.9^PB(5)+.2^Pb(6) + .2*PBC8) 
AC6)^.3itPBC3) + .15i^PB(5)+.9itPBC6)+.3*PBC9) 

AC7) = .2^PBC9) + .AitPBC7)+.2i'PBC8) 
AC8)=.15i^PBC5)+.3i^PBC7)+.9i^PBC8)+.3itPBC9) 
AC9)=.2i^PBC6)+.2i^PBC8) + .9*PB(9) 

DO 7 2 IK^ 1 . NN 
PBC IK ) ^AC IK ) 

72 CONTINUE 

71 CONTINUE 

PbC SC T ) ) =0 

prar o . o 

DO 51 K=1.NN 

PfJD = Pr^D + PBC K) 

51 corMimjE 

DO 9 2 K-1 . FiN 
P b ( K ) - B B C K ) 

92 CONTINUE 
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INPUT DATA 



RETURN 
END 
C 
C 

c 

15 9 

12585658565 

393959393 

129 

1235 

236 

1957 

29568 

3569 

978 

5789 

689 

110 10 0 
1110 10 



0 0 10 0 



110 10 0 



0 10 1110 



0 
0 

0 0 0 
0 0 0 



10 110 



0 10 0 1 



1 0 
0 1 
1 0 



10 111 
0 10 11 
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APPENCIX B 

FOBTfiAN CODE OF F-H ALGORIIHl 



If^TEGER TIME,T,N,IN(9),A(9,9),S(10),CELL.Z(11) 
REAL R(9,9),X(9,10),V(9,11) , PND , BB ( 9 ) , P B ( 9 ) 
REAL C(9.10),XMIN,P(9,10) 

^ INPUT ^ 

T=TIME PERIODS 
N=«0F CELLS 
S^STARTING PATH 
If^ = fiOF ADJACENT CELLS 
A = ADJACEfJT CELLS 
R^TRANSITION MATRIX 

READ(5,1G0) T,N 

100 FGRMAT(2I2) 

READ(5,101) (SCI), 1=1, T) 

101 FORMAT(lOIl) 

READ(5,102) (IN(I),I=1,9) 

102 F0RMAT(9I1) 

READC5, 105) ( ( A ( I , J ) , J = 1 , N ) , I = 1 , N ) 

105 F0RMAT(9I1) 

READ(5, 109) ( (R( I, J), J = 1 .N), 1=1 ,N) 

109 F0RMAT(9F5.2) 

DETERMINE V(I,J)'S 

CALL VFIND(S,T,f^ V) 

DETERMINE P(I,J)*S 

CALL PFIND(S,T,P) 

DO 15 I=1,N 
PB( I ) = P( I , 1 ) 

15 BB(I)=P(I,1) 

DETERMUJE X(I,J)’S ^ ^ ^ 

CALL XFIf^DC S, T , r^, X ) 

Fir^D PND FOR A STARTir^G PATH it 

CAl L PROB(S,PB,T,N,BB.Pf^D) 

XXMirj = Pt^D 

He * a * * FIND t^EN EXTREME POlfJT ^ ^ Hi * ^ * ^ ^ *: * 

Z( 1 ) = 1 

999 CALL CFIND(2,Z( 1 ), IN,X,R.V,A,P,CELL) 

Z(2)=CELL 

CALL CFirJD(5,Z(2),IN,X,R.V,A,P,CELL) 

Z( 5 ) ^CEL L 

CALL CFIND(5,Z(5),IN,X,R,V,A,P.CELL) 

Z(9)=CELL 

CALL CFIf^D(9,Z(9),IN,X,R,V,A,P,CELL) 
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Z(5)=CELL 

CALL CFIND(5,Z(5),IN,X,R,V,A,P,CELL) 

Z(6 )^CELL 

CALL CFIfJD(D,Z(6 ) . IN,X.R, V,A,r ,CELL) 

Z(7 ):^CELL 

CALL CFINDC7 ,Z(7 ) , I N , X , R , V , A , P , CEL L ) 
Z(8)=CELL 

CALL CFIND(8,Z(8), IN,X,R,V, A,P,CELL) 
Z(9)=CELL 

CALL CFIND(9,Z(9) , IN,X,R,V,A,P,CELL) 

Z( 10)=CELL 

CALL CFINDC 10,Z( 10) , IN,X,R, V, A,P,CELL) 

Z( 11 )=CELL 

FIND PND FOR A NEN EXTREME POINT 

CALL PROB(Z,PB,T,N, BB,PND) 

COMPARE TNO EXTREME POINT’S PND 

1FCPND.lt. XXMIN) THEN DO 
XXMIt^ = PND 
DO 90 1=1 ,T 
90 5(1)=Z(1+1) 

DETERMINE NEN V(I,J)’S, P(1,J)’S, X(1,J)’S 

CALL VF1ND(S,T,N, V) 

CALL PF1ND(S,T,P) 

DO 11 1=1 ,N 
PB( 1)=P( 1 , 1 ) 

11 BB(1)=P(1,1) 

CALL XF1ND(S,T,N,X) 

GO TO 999 
END IF 

IF THE NEN EXTREME POINT DOES NOT IMPROVE > 
the objective function, STOP 

500 PRINT, ’MIN PND IS’, XXMIN 

PRINT, ’ OPTIMAL PATH I S ^ ^ ’ 

NRITEC6 ,600 ) (SC I ) , 1 = 1 , T ) 

600 FORMATC 3X, 1012 ) 

STOP 

END 



^ CALCULATES V(I,J)’S 



SUBROUTINE V F I MD ( S , T , N , V ) 
REAL V(9,ll ) 

IfITEGER SC 10) ,T 
DO 15 I=1,N 



15 




VC I 




J = T 
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K = 


JU 




VC 1 


, J ) 




VC2 


, J ) 




VC 3 


, J ) 




VC A 


, J ) 




VC 5 


, J) 




VC 5 


, J) 




VC6 


, J ) 




VC7 


, J ) 




VC 8 


, J) 




VC 9 


, J ) 




VCS 


C J ) , 




J = J 


-1 



, 11 ) = 1 .0 



= .A^VCl ,K)+ . 
= .2i(V(l,K)+. 

= .3^^VC2,K)+. 
= .2*V( 1 ,K)+ . 

= .15*V(2,K)+ 

= VC5, J))(( .15 
= .2‘^V(3,K)+. 

= .3*V(A,K)+. 

= .2^V(5,K)+. 
= .3^^VC6,K)+. 

J ) = 0 



3i^V(2,K)+ .3^VCA,K) 
A*VC2,K)+.2^V(3,K)+.2*V(5,K) 
A»V(3,K)4 .3»V(6,K) 
A*V(A,K)+.2^V(5,K)+.2*VC7,K) 
.15^V(A,K)+.9*V(5.K) 

* V ( 6 , K ) + . 1 5 V ( 8 , K ) ) 

2^V(5.K) + .9i(V(6,K)+ .2^VC9,K) 
A ( 7 , K ) + . 3 V ( 8 . K ) 
2^VC7,K)+.A*V(8,K)+.2*V(9,K) 
3^VC8,K)+ . A*V(9,K) 



IFCJ.GT.O) GO TO 25 
RETURN 
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END 



CALCULATES COST COEFFICIENTS AND 
SOLVES LP SUBPKObLEM 

SUBROUTINE CFl ND( T1 ME , MM , 1 N , X . R , V . A , P , CEL L) 

REAL R(9,9),V(9,11),P(9,10),PP,CMIN,X(9,10).C(9,10) 
INTEGER TTIME,TIME,MM, IN(9),A(9,9) ,CELL ,MI ,MK( 10) ,LL 
CMIN=1 
IK=IN(MM) 

TTIME=TIME+1 
DO ^40 1 = 1, IK 
J=A(MM, I ) 

PP = 0 . 0 
IB=IN( J) 

IF(P( J,TIME) .EQ.O) THEN DO 
DO 55 11 = 1, IB 

PP = PP + R( J,A( J,II ) ))^V(A( J,II),TTIME) 

55 CONTINUE 

C(MM, J) = -( 0.0001 )^(PP 
ELSE DO 

DO 56 11 = 1 , IB 

PP=PP+R( J,A( J,I1 ) )^V(A( J, II ),TTIME) 

56 CONTINUE 
C(MM,J)=-(P( J,T1M£)^PP) 

END IF 

IF(C(MM. J) .LT.CMIN) THEN DO 
CM1N=C(MM, J) 

CELL=J 
END IF 

AO CONTINUE 
RETURN 
END 

CALCULATES X(I,J)*S 

SUBROUTINE XF 1 ND ( S , T , N , X ) 

INTEGER S(10),T 
REAL X(9,10) 

DO 60 1=1 ,T 
DO 7 0 J = 1 ,N 
X( J , 1) = 0 . 0 
70 corjTir^uE 

60 CONTirJUE 

DO 78 1=1, T 

78 X(S( 1 ), I)=l .0 

RETURN 
END 

CALCULATES PfJD 

SUBROUTINE PROBC S , PB , T . N . BB , PfJD ) 

REAL AA( 50 ) . PB( 9 ) , PND, BBC 9 ) 
irnEGER N,T,S( 10) 

KP=T-1 

DO 71 K=1.KP 
PBCSCK) ) = 0 

AA(l)=.A^^PB(l)+.2?fpB(2)+.?^PB(A) 

AA(2)=.3*PB(1)+.A^^PB(2)*-.3«PB(3)+.15^^PB(5) 

AA(3)=.2^PB(?)+.A#PB(3)+.2*PB(b) 

AA(A)=.3*PB(1)+.A^PB(A)+.15>PBC5)+.3M^B(7) 

AA(5) = .2^PB(2)+.2^^PB(A)+,A*PB(5)+.2*PB(6)+.2^^PB(8) 

AA(6)=.3*PB(3)+.15*PB(5)+A^PB(6)+.3^PB(9) 

AA(7)=.2*PB(A)+.A^^PB(7)<-.2^PB(8) 

AA(8) = .15*PB(5)+.3>Pb(7)+.A^PB(8)+.3«^PB(9) 
AA(9)=.2^PB(6)+.2^PB(8)+.A-PB(9) 

DO 72 1K=1 , N 
PBC IK ) =AA( IK) 

72 CONTINUE 
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71 CONTINUE 

PB(SCT) ) = 0 
PfJD = 0 . 0 
DO 50 K = 1 , N 

PND=PND+PB( K) 

50 CONTINUE 

DO ^2 K=1 ,N 
PB(K)=BB(K) 

CONTINUE 

RETURN 

END 

C 

C CALCULATES P(I,J)'S 

C 

SUBROUTINE PFIND(S,T,P) 

REAL P(9,10),PM(9,10) 

INTEGER S( 10) ,T,M 
P(9,l)=l .0 
PMC 9 , 1 ) = 1 . 0 
DO 85 1=1,8 
PMC I , 1 ) = 0 

85 PCI,1)=0 
DO 80 J=2,T 
M=J-1 

PMC5CM) ,M)=0.0 

PC1,J) = .A)(PMC1,M)+.2^^PMC2,M)+.2^PMCA,M) 

PC 2, J ) = . 3^PMC 1 ,M) + . A^PMC 2,M) + . S^^PMC 3 .M)+ . 15)(PMC 5 ,M) 
PC3, J) = .2^PMC2,M) + . A^PMC 3,M) + .2^^PMC6,M) 

PC A, J ) = . 3)^PMC 1 ,M) + . A^PMC A ,M) + . IS^^PMC 5 ,M) + . 3^^PMC7 ,M) 
PC 5, J ) = .2*PMC2,M) + .2^tPMC A,M) + . A^^PMC 5,M) + . 2<PMC6 ,M) 
PC5, J)=PC5, J)^^.2^^PMC8,M) 

PC6, J) = .3^PMC3,M)+. 15^PMC5,M)+.A^^PMC6,M) + .3^PMC9,M) 
PC7,J)=.2^PMCA,M)+.A^PMC7,M)+.2XPMC£,M) 

PC8,J) = .15^PMC5,M)+.3^PMC7,M) + .A^^PMC8,M)+.3^PMC9,M) 
PC9, J) = .2)(PMC6,M)+.2)^PMC8,M) + . A^PMC9,M) 

DO 86 1=1,9 

86 PMC I, J)=PCI, J) 

80 CONTINUE 

RETURN 

END 

C 

C INPUT DATA 

C 

10 9 

A565856585 

3A3A5A3A3 

12A000000 

123500000 

236000000 

1 A5700000 

2A5680000 

356900000 

A78000000 

5/8900000 

689000000 

OA0030000030000000000000000 
0200A00200 00020000000000000 
0000300 A 0000 0 00030000000000 
0200000 OUOAU020000U2000 0000 
0000150000150A0015000015000 
00 00000200000 2 OOAO 000000020 
0000000000300000000A0030000 
000000 00000 00200000200A 0020 
0000000000 000000300000300AO 
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